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Abstract. Nancy Flournoy was born in Long Beach, California, on 
May 4, 1947. After graduating from Polytechnic School in Pasadena in 
1965, she earned a B.S. (1969) and M.S. (1971) in biostatistics from 
UCLA. Between her bachelors and masters degrees, she worked as a 
Statistician I for Regional Medical Programs at UCLA. After receiving 
her master’s degree, she spend three years at the Southwest Laboratory 
for Education Research and Development in Seal Beach, California. 
Flournoy joined the Seattle team pioneering bone marrow transplan¬ 
tation in 1973. She moved with the transplant team into the newly 
formed Fred Hutchinson Cancer Research Center in 1975 as Director 
of Clinical Statistics, where she supervised a group responsible for the 
design and analysis of about 80 simultaneous clinical trials. To support 
the Clinical Division, she supervised the development of an interdisci¬ 
plinary shared data software system. She recruited Leonard B. Hearne 
to create this database management system in 1975 (and married him 
in 1978). While at the Cancer Center, she was also at the University 
of Washington, where she received her doctorate in biomathematics in 
1982. She became the first female director of the program in statistics at 
the National Science Foundation (NSF) in 1986. She received service 
awards from the NSF in 1988 and the National Institute of Statisti¬ 
cal Science in 2006 for facilitating interdisciplinary research. Flournoy 
joined the Department of Mathematics and Statistics at American Uni¬ 
versity in 1988. She moved as department chair to the University of 
Missouri in 2002, where she became Curators’ Distinguished Professor 
in 2012. 

While at the Cancer Center, Flournoy documented the graft-versus- 
leukemia effect in humans and discovered a source of frequent lethal 
viral infections in the bone marrow transplant patients. Later she was 
influential in developing adaptive experimental designs. Her numerous 
honors include fellow of the Institute of Mathematical Statistics (1990), 
the American Statistical Association (1992), the World Academy of 
Arts and Sciences (1992) and the American Academy for the Advance¬ 
ment of Science (1993). She has received the COPSS Scott (2000) and 
David (2007) awards, and the Norwood (2012) award from the Univer¬ 
sity of Alabama. 
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Fig. 1. Nancy at home in a part of Los Angeles County that 
was then ealled Potero Heights, 1949. 

1. EARLY LIFE 

Rosenberger: Tell us a little about your early life. 
Where did you grow up and what did your parents 
do? 

Flournoy: I was born in Long Beach, CA, and grew 
up in Los Angeles County in a lemon orchard sur¬ 
rounded by oil wells and a flood plain. There was a 
dairy farm nearby and we had a donkey. My father 
was a plumbing contractor who plumbed Los An¬ 
geles; restaurants, dormitories, cemeteries. He had 
11 trucks go out every day. My Mom was always 
unhappy about not finishing college, so she enrolled 
in college when I went to college and then directed 
a preschool for many years. I have three brothers 
and one sister. I was sent to Polytechnic School in 
Pasadena as a sophomore in high school. On the en¬ 
trance exam I had the second highest score in math 
in history, but I flunked the English exam because I 
didn’t know the words in the instructions. (So even 
then I had a one-sided brain!) 

Rosenberger: As a young person, were you inter¬ 
ested in mathematics, statistics, data? What made 
you excited about statistics? 

Flournoy: High school algebra really made me 
happy; I would lay on the floor and work problems 
for hours. I had a new female instructor whose hus¬ 
band had gotten a professorship across the street at 



Fig. 2. Nancy at her graduation from Polytechnic School, 
Pasadena, 1965. 

Cal Tech while she just landed a high school job; 
her anger came through and I got the message that 
mathematics is worth being passionate about. 

My love of statistics came as a junior at UCLA, 
when I took a course taught by Don Ylvisaker. I just 
assumed that Don was a great teacher for all time, 
but he later told me that he never had another class 
like it. Four or five students from that class went on 
to get doctorates in statistics. 

Rosenberger: You were fortunate to be at UCLA 
at a time when there were some of the great names in 
biostatistics: Abdelmonen Afifi, Frank Massey, Wil 
Dixon, Olive Dunn, Virginia Clark. What professors 
excited you at UCLA? 

Flournoy: Afifi was the young dynamic professor 
and taught out of Scheffe; all the students loved 
Afifi. Dixon had a bimodal distribution among the 
students; you either loved him or hated him. He put 
out a thousand ideas a minute; if you paid close at¬ 
tention, you would find they were pearls. It was a 
challenge to get what he was saying as he didn’t 
change the tone of his voice when he switched from 
one topic to another. He taught the power of data 








A CONVERSATION WITH NANCY FLOURNOY 


3 


analysis as a tool for learning and a thousand lit¬ 
tle ways to make the data sing. I had a class with 
Frank Massey; I learned a lot, but he was quiet and 
not dynamic. 

Rosenberger: Did you have any connection to the 
Department of Statistics? You mentioned Ylvisaker. 
What about Paul Hoel? 

Flournoy: A separate statistics department did 
not exist at that time; it was a math department 
with a few statisticians. I used the Hoel, Port and 
Stone probability book when it was just a set of 
notes. I don’t think Hoel was the instructor though. 

Rosenberger: What interested you in biostatis¬ 
tics? 

Flournoy: Most of the statistics courses that were 
offered at UCLA were in the Department of Bio¬ 
statistics. Prior to taking statistics, I had loved bio¬ 
chemistry and was a nutrition major, leading to my 
major in the School of Public Health (SPA). When 
1 recognized that with a degree in nutrition, I would 
probably only be able to run a cafeteria in a hos¬ 
pital, I decided to get my degree in mathematics 
instead. I applied repeatedly to change from SPA to 
the College of Arts and Sciences (CAS), but my ap¬ 
plication would get turned down. In tears, I didn’t 
know what to do. Then the SPA Dean asked why 
I was flunking out, which didn’t make sense since 
I always had gotten As and Bs. They had lost all 


my records because I had changed names when I 
was previously married, and nothing followed me. 
So that’s why they didn’t accept me at CAS. By 
the time this got settled, I had enough credits to 
get a degree in biostatistics. 

2. GRADUATE SCHOOL 

Rosenberger: What did you do after you gradu¬ 
ated? How did you get to graduate school? 

Flournoy: When I got a job at Regional Medi¬ 
cal Programs as a Statistician I, one old man would 
come around and ask me to add numbers; I told him 
he could hire a statistical clerk for half my salary. 
I was told that, as a young woman, my presenta¬ 
tions were not credible. So they hired a male DrPh 
to present my reports in his name. As a mild way 
of protesting, I put my hair in a bun, dyed it white, 
and got hred. They said I was an “uppity woman.” 
At that time, Virginia Clark was department chair. 
She said, “We have a fellowship, why don’t you come 
to grad school?” I have some happy memories of my 
master’s program at UCLA: Olive Dunn supervised 
my master’s thesis; Mary Ann Hill was a great teach¬ 
ing assistant for Dixon’s class; Carol Newton taught 
a mean FORTRAN programming course; and Ray 
and Jean Mickey were influential in my career deci¬ 
sions. 



Fig. 3. Nancy with her parents, Elizabeth Blincoe and Carr Irvine Flournoy, at her graduation from the University of 
Washington in 1982. 
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Rosenberger: When you won the David Award, 
you talked about meeting F. N. David. Tell us about 
that. 

Flournoy: I was in the Los Angeles chapter of the 
ASA; around 1972, a group of us carpooled out to 
UC Riverside where David was giving a talk. She 
had a strong presence, standing with one leg up on 
a stairstep and smoking a cigar while she talked. 
It was a roomful of people, and she exuded such 
confidence. So I immediately started smoking cigars. 
I had been used to seeing female statisticians such as 
Clark and Dunn behind a desk and not commanding 
an audience. 

Rosenberger: How did you get to University of 
Washington (UW)? 

Flournoy: After the M.S., I thought I knew every¬ 
thing about statistics. I got a job at Southwest Ed¬ 
ucation and Laboratory for Research, where there 
were a lot of education psychologists who were into 
experimental design. On my second day, they pre¬ 
sented me with computer output that had more than 
one error term; I had the good sense to keep my 
mouth shut. I immediately called Wil Dixon and 
asked what they were talking about. He replied, “Oh 
well, we can’t teach you everything.” He suggested 
I get a book by Walt Federer. The book was out of 
print, but Walt got a preprint from India and sent it 
to me; so I spent my nights reading Federer’s book. 

Later, I was trying to read the Journal of the 
American Statistical Association to implement some 
of the stuff I wanted to do, and I found I couldn’t 
read the literature. I also wanted to escape the smog 
of Los Angeles. So I applied to the UW, my only 
application. Dick Kronmal said there was a research 
assistant position with the bone marrow transplant 
team, which was then located in the Old Public Hos¬ 
pital (recently Amazon) in Seattle. 

At that time, there was no statistics department 
at UW. The mathematical statistics courses were 
taught in the Department of Mathematics. I took 
the mathematical statistics sequence from Galen 
Shorack. I had courses from Ron Pyke and Fritz 
Schultz in nonparametrics. Shortly after Fritz left 
for Boeing, the remaining statistics faculty formed 
the Department of Statistics. In the Department of 
Biostatistics, there were some female faculty: Paula 
Diehr and Pat Wahl. I took the first categorical data 
analysis class taught at UW from Norman Breslow. 
He gave quizzes at the end of class, so I never paid 
so much attention in a course before. I took survival 


from Ross Prentice early in the days of the Cox pro¬ 
portional hazards model. 

Rosenberger: What was it like working with your 
dissertation advisor, Lloyd Fisher? 

Flournoy: It worked out well because we have sim¬ 
ilar work styles. Both of us had busy consulting lives; 
we would schedule meetings and get our business 
done. 

3. THE SEATTLE BONE MARROW 
TRANSPLANTATION TEAM 

Rosenberger: Today every street corner seems to 
have a contract research organization for data co¬ 
ordinating centers on large clinical trials. But when 
you went to the Fred Hutchinson Cancer Research 
Center, information technology was primitive, such 
places did not exist. You had to create that envi¬ 
ronment on your own. What was it like? What were 
the challenges? 

Flournoy: That’s an interesting story. Dick Kron¬ 
mal had invested a lot of effort in creating a database 
management system without requiring a rectangular 
data structure. Updates required physically sorting 
the cards (remember all data records had to fit into 
the 80 digits of a Hollerith punch card—so I tend to 
use the words “card” and “record” interchangeably). 
There was a transplant data set in place with seven 
different kinds of cards. Kronmal had E. Donnal 
“Don” Thomas (Director of the Clinical Research 
Division at the Cancer Center) buy a computer. The 
computer weighed 50 pounds (I could toss it in my 
van and take it home; the cost was about $50,000), 
and data storage was on Phillips cassette tapes. 
Records could be transmitted across the phone wires 
and then integrated into the database at UW. Ini¬ 
tially, there was not much data (only 10 patients) 
and the first update took my whole computing bud¬ 
get for the year! What I did then for some period of 
time was, when it was time to do an update, punch 
cards of the whole database and the new dataset; I 
would physically sort and merge the cards by hand 
and load them into SPSS. That was my “dirty laun¬ 
dry” story because the laundromat had big long ta¬ 
bles and I sorted cards while doing laundry. Kronmal 
told me that, if I had any trouble with my new com¬ 
puter, I should call Leonard Hearne. Index sequen¬ 
tial files were brand new at that time, and Leonard 
used them to create an early database management 
system before the word was in the literature (see 
Flournoy and Hearne, 1981, 1990a, 1990b). We used 
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it for several years until a commercial system came 
on the market. At site visits, someone would ask a 
question and I would pass a note down to a program¬ 
mer, who would extract the answer in 15 minutes or 
so. We set the bar for oncology programs. 

Ross Prentice came from the University of Wa¬ 
terloo with a box of cards on the Cox proportional 
hazards model; we were really early using that. The 
doctors were smart enough to understand the limi¬ 
tations in using discriminant analysis and they were 
thrilled to be able to incorporate censored survival 
data in their regression models. My work document¬ 
ing graft vs. leukemia in humans was very impor¬ 
tant (see Weiden et ah, 1979, 1981a, 1981b, 1981c). 
One hypothesis motivating the development of bone 
marrow transplantation was that the marrow graft 
would attack residual leukemia also. Immunological 
activity of the graft was apparent when the graft 
instigated an immunological attack on the patient. 
I modeled the impact of this attack on the relapse 
rate. The protection of the graft attack against re¬ 
lapse greatly complicated post-transplant treatment 
strategies. But our findings have withstood the test 
of time. It was, perhaps, the first major applica¬ 
tion of the proportional hazards model with time- 
dependent covariates. 

Rosenberger: When you think of the success of the 
bone marrow transplant program (Don Thomas won 
the Nobel prize in 1990 for developing bone marrow 
transplantation as a treatment for leukemia), how 
much did statistics and data management play a 


role in that? Do you think statistics and data man¬ 
agement will ever get its due? 

Flournoy: We had this rudimentary set of records 
that could be added onto infinitely. It started out 
that bacteriology wanted to add a card, then vi¬ 
rology, then specific studies would add a card with 
their data. Before you knew it, we had an inter¬ 
disciplinary shared database with assigned patient 
numbers so all the integrated data was available. 
I was able to say “do you know what they’re do¬ 
ing in virology that’s related” because I knew ev¬ 
erybody’s data. It wasn’t until many years later 
that people started talking about having integrated 
shared databases. Most were established for billing 
purposes, not for research purposes. They are differ¬ 
ent constructs. Hospitals would archive data after 
the bill was paid but we wanted to keep it around 
forever. 

When the program started, there was one of ev¬ 
erybody (one statistician, one virologist, etc.), and 
we would sit around the table and share results. It 
was important to be influential and to catch prob¬ 
lems in data collection and quality control before 
they got big. When working with new doctors, there 
were humps you had to get over because they would 
claim that there were no quality control issues: their 
lab people never made a mistake. A lot of negotia¬ 
tion had to go on before we could agree. Yes, we had 
a huge influence. Even randomization and blinding 
was controversial. If it was in the middle of the night 
the cards might get shuffled; there was too much 
room for bias. We introduced them to a very careful 



Fig. 4. Yash Mittal, first female director of the probability program, and Nancy, first female director of the statistics program, 
at NSF. 
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randomization regimen for treatment assignments, 
with a 24 hour on-call person. 

It will be hard for statistics and data manage¬ 
ment to ever get its full due because the doctors 
are so enamored of themselves (laughs). It’s really a 
strange system where the people with the least sci¬ 
ence background usually run the science. Also, the 
data management budget was always the first to be 
cut; yet it is very expensive to do a quality job. 

Rosenberger: What has your role been in fostering 
interdisciplinary research? 

Flournoy: Having conducted interdisciplinary re¬ 
search for more than a decade at the Cancer Center, 
I knew the power that teams of interdisciplinary 
researchers could bring to bear on important sci¬ 
entific questions. Coincidentally, when I went to 
the National Science Foundation (NSF) in 1986, 
the Division of Mathematical Sciences (DMS) had 
funded the Institute of Mathematical Statistics 
(IMS) to write a report on cross-disciplinary re¬ 
search. I watched the growth in their thinking as 
they interacted with each other. At the time, the 
discipline did not appreciate the role of applications 
in academic settings. I think I was able to influ¬ 
ence the IMS cross disciplinary committee on the 
valuable nature of interdisciplinary work. The re¬ 
port of the committee had a dramatic effect on the 
discipline. The report proposed establishing the Na¬ 
tional Institute of Statistical Science. Since I was at 
NSF, I was able to promote the idea of establishing 
a broad institute that would work on problems of 
national importance. 

At the same time, I would receive proposals from 
statisticians motivated by applications. Because our 
budget was small, I took such proposals around to 
the relevant disciplines that were involved, and was 
able to get some joint funding. This resulted in my 
getting an award in 1988 for facilitating the fund¬ 
ing of these interdisciplinary projects. This also led 
to specific DMS requests for proposals for interdis¬ 
ciplinary research projects, which are now common 
throughout NSF. 

4. ADAPTIVE DESIGNS 

Rosenberger: How did you get interested in adap¬ 
tive designs? 

Flournoy: While at the Cancer Center, the major 
program project grant had five-year reviews. When 
we prepared for the third one of these, we spent a 
year reviewing what we had done and how we would 


go forward. In the course of that retrospective, I 
developed some feelings about the two arm clinical 
trial. The standard ideas about the two arm clini¬ 
cal trials came from the Peto paper in the mid 70s 
(Peto et ah, 1976, 1977). But, in my experience, a 
treatment is a point in a high dimensional space: in¬ 
volving drugs, radiation, including how much, how 
often; and one learned little about this high dimen¬ 
sional space using the traditional two arm clinical 
trial. For instance, we spent five years comparing 
A to B; but then we go back to the high dimen¬ 
sional space and pick out point C, and have another 
five years of experimentation and compare A to C. 
Then we compare C to D, and after 15 years we 
have knowledge of four points in a high dimensional 
space. I believed it would be more efficient and in¬ 
formative to know which direction we should head 
in the high dimensional space. So that led me to 
think about adaptive designs. I recommended sev¬ 
eral to the group and the physicians liked the ideas, 
but thought they may be too radical to get funded. 

Another thing that promoted my interest was 
looking at pilot studies to decide what to take for¬ 
ward to larger studies. Bob Tsutukawa was visiting 
the Cancer Center from the University of Missouri 
at the time. I thought his Bayesian ideas were ap¬ 
pealing and I used expert opinion for prior elicita¬ 
tion (see Flournoy, 1993). The prior was way off, so 
we wound up with a lot of toxicities. You just can’t 
trust the best expert opinion of the best experts, 
and so there needed to be some way to use interim 
data faster to adapt and put much less weight on 
the prior. My later work showed how random walk 
rules could be constructed to do this (see Durham 
and Flournoy, 1994; Durham, Flournoy and Rosen¬ 
berger, 1997; Flournoy and Oron, 2015). 

Rosenberger: The first time I heard the name 
Nancy Flournoy was in the context of the 1989 ses¬ 
sion on adaptive designs at Joint Statistical Meet¬ 
ings (JSM) in Washington. It turned out to be one 
of the most controversial sessions in the history of 
JSM. Talk about that. 

Flournoy: My experience at NSF was that you 
don’t make progress without community. One per¬ 
son alone doesn’t get much done. So I had the idea 
that a JSM session on adaptive design would bring 
together people who are interested in adaptive de¬ 
signs. I didn’t know anyone personally. I invited 
based on my impressions of their interests. I in¬ 
vited Don Berry, Richard Simon and Janis Hard¬ 
wick. I gave a straightforward technical talk on the 
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Fig. 5. 1994 Workshop on Sequential Analysis at University of North Carolina, Chapel Hill. Included are Nancy 

Flournoy (seated second from left), Lynne Hillard (immediately behind Nancy), Janis Hardwick (to the right of Lynne), Bill 
Rosenberger (third row from back, middle), and Steve Durham (directly in front of right window). 


topic. The remainder of the session focused primar¬ 
ily on criticism of the extracorporeal membrane oxy¬ 
genation (ECHO) trials (e.g., Barlett et ah, 1985; 
O’Rourke et ah, 1989; Ware, 1989). (The ECHO 
trial was an implementation of the randomized play- 
the-winner rule of Wei and Durham, 1978, in which 
11 babies were assigned to an experimental arm, and 
all survived, while one baby assigned to the conven¬ 
tional arm, died. The historical death rate on the 
conventional arm was 80 percent.) To my dismay, 
all the negative focus of the session was directed to¬ 
ward the adaptive design aspect of the clinical trial, 
rather than on the sample size and what kind of 
sample size would be needed for the trial to be con¬ 
vincing. The press that was generated by this session 
set adaptive designs back a long time. 

Rosenberger: How much do you think the failed 
ECMO trial inhibited the development of adaptive 
designs? 


Flournoy: What would have been a reasonable ap¬ 
proach? The original trial was unconvincing due to 
having few patients, in spite of the fact that a prob¬ 
abilistically reasonable stopping rule was applied. 
The controversy over the subsequent two arm trial in 
clinical research set back adaptive designs wrongly. 
The adaptive trial was so successful that only one 
baby died; is that bad? 

Rosenberger: In your 1992 AMS/IMS/SIAM con¬ 
ference on adaptive designs (Flournoy and Rosen¬ 
berger, 1995), you brought together some of the 
groundbreakers of adaptive designs along with a 
number of younger faculty who are now at the fore¬ 
front of the discipline. At the opening session, you 
started by talking about the need to streamline the 
process of clinical trials, the end to phases and the 
incorporation of dynamic interim decisions. You said 
that will revolutionize the way we do clinical tri¬ 
als, and that this conference would be an ambi¬ 
tious beginning to that revolution. Now, over two 
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decades later, there are 70-some sessions on adap¬ 
tive designs at the Joint Statistical Meetings, “big- 
pharma” working groups, Food and Drug Adminis¬ 
tration white papers and guidelines, companies like 
ADDPLAN, and CYTEL devoted to adaptive de¬ 
sign software and everyone wants to do adaptive de¬ 
signs. What took so long? 

Flournoy: ECHO made a steep hole to climb. We 
also had to develop theory. It was one thing to say 
“this is a good idea,” and another to adequately 
support it. Some ideas were NOT good. This in¬ 
cludes a class of procedures that derive from stochas¬ 
tic approximation, that Val Eedorov coined “best 
intention” designs. In these designs, a target dose 
is estimated (such as the dose having a particular 
percent toxicity or one that maximizes some utility 
function); then that estimate is the dose given to 
the next subject. Some, including Lai and Robbins 
(1982), understood early on that using this proce¬ 
dure without safeguards may result in treatment se¬ 
quences that converge to the wrong dose. But others, 
including myself (Li, Durham and Elournoy, 1995), 
were enamored of this idea and ignorant of ear¬ 
lier warnings. This approach remains popular today 
even as recent publications are exposing just how 
misleading it can be (e.g., Azriel, 2012, Oron and 
Hoff, 2013). 

In the 1980s, John Whitehead spent a year visiting 
the Cancer Center from the University of Reading, 
and promoted the idea of using sequential stopping 
rules taking censoring into account. Its value was so 
obvious that I expected that by 1990 every clinical 
trial would be using these techniques. So I focused 
instead on adaptive allocation. At American Uni¬ 
versity (AU), I worked on theoretical problems in 
these areas. When I pulled my head out and looked 
around I was shocked to see that stopping rules in¬ 
corporating censoring were not being used, except a 
bit in cancer. So things that seem obvious to some 
can take a long time to enter the medical arena. 
Take the “3 -|- 3” dose escalation design as an exam¬ 
ple. It has been soundly discredited (Reiner, Pao- 
letti and O’Quigley, 1999; Lin and Shih, 2001), and 
yet remains a standard practice in oncology phase I 
trials. 

Adaptive allocation is still in its infancy compared 
to sequential monitoring and stopping. Now there 
has developed a new belief that simulation is ad¬ 
equate for assessing an adaptive design. But rely¬ 
ing solely on simulation muddies the water because 
there is no global view of what is driving the design. 


In addition, there are many papers in the literature 
that report only averages over simulations without 
measures of variability. When you consider measures 
of variability, a completely different picture emerges 
(Oron and Hoff, 2013). I fervently believe in devel¬ 
oping the theory underlying classes of designs. Eor- 
tunately, many people are interested in working on 
the theoretical challenges, and there are a lot of in¬ 
teresting open questions. 

Rosenberger: Many times when I hear talks on 
adaptive designs I want to scream out “Nancy 
Flournoy thought of that in the 1980s.” How do you 
feel about some of your early ideas being ignored? 

Flournoy: Well, I’m hardly alone in this. For in¬ 
stance, Chris Jennison invented many clever tech¬ 
niques for sequential and adaptive clinical trials 
very early that are sometimes “rediscovered” with¬ 
out reference (e.g., Jennison, Johnstone and Turn- 
bull, 1982; Kulkarni and Jennison, 1986; Jennison, 
1987). In my case, it amazes me that there are a 
large number of people who will reference a paper 
from the 1980s and ignore 30 years of my research. 
For example, the early up and down paper of Storer 
(1989) is often cited without reference to my later 
papers that have much more sophisticated control 
of the adaptive process. This early paper is used as 
a whipping post to declare up and down procedures 
inferior. An up and down design is a random walk 
that can end anywhere. The last state (dose) vis¬ 
ited should not be used as an estimator. But this 
is done when the up and down design is compared 
to other procedures that derive from stochastic ap¬ 
proximation (e.g., Zacks, 2009). That bothers me a 
lot. 

Rosenberger: How did you meet Steve Durham? 
This began one of the great collaborations in statis¬ 
tics. Tell us about that. 

Flournoy: One of the few positive consequences of 
the 1989 JSM session was meeting Steve Durham 
from the University of South Carolina. When I 
walked out the door after the session, Steve intro¬ 
duced himself and was very excited because we were 
basically working on the same mathematical prob¬ 
lems, his from an engineering motivation, and mine 
from a medical motivation. We began working to¬ 
gether right away. He would come to Washington, 
DC, to meet me, and I went to South Carolina. After 
a stint as Chair at AU, I was on a sabbatical at the 
University of North Carolina Chapel Hill; Leonard 
and I bought a house close to campus so that we 
could host visitors. In particular, Steve Durham and 


A CONVERSATION WITH NANCY FLOURNOY 


9 


I worked together quite a lot in that house and at 
the Department of Statistics. Several other collab¬ 
orators came down for extended periods, including 
you (W.F.R.) and two of my doctoral students: Eloi 
Kpamagen (now at Novavax) and Misrak Gezmu 
(now at National Institutes of Health). 

Rosenberger: The introduction of the random 
walk rules coincided with the introduction of the 
continual reassessment method (CRM; O’Quigley, 
Pepe, and Fisher, 1990) in the Bayesian context. In 
particular, you and Steve worked out the entire ex¬ 
act distribution theory of a class of designs, while 
others were relying on simulation. How does this 
rank in terms of your contributions to statistics? 

Flournoy: The random walk rules are extremely 
practical and mathematically elegant, so it was a 
lot of fun to develop the theory. They are the stan¬ 
dard in many areas of science (e.g., American So¬ 
ciety for Testing and Materials, 2010; Treutwein, 
1995; National Institute of Environmental Health 
Sciences, 2001). The key property that we discovered 
was how to control the allocation coverage by intro¬ 
ducing an appropriate bias (Durham and Flournoy, 
1994). Steve was always thinking in terms of engi¬ 
neering applications; I was always thinking in the 
dose-response context. We did “reverse engineer¬ 
ing,” in that we had a target allocation in mind, and 
we found design parameters to facilitate this. The 
designs are nonparametric in that allocation does 
not depend on estimates of model parameters. They 
are extraordinarily simple to illustrate and have ex¬ 
act distribution theory that is unavailable for other, 
more complicated designs. 

Rosenberger: Some have lumped random walk 
rules in the context of generic dose escalation de¬ 
signs, such as the 3-1-3 design, that has no op¬ 
timal properties. At the same time, Bayesian ap¬ 
proaches, such as the CRM were becoming increas¬ 
ingly well-known. Talk about the historic interplay 
among these approaches. 

Flournoy: Lloyd Fisher and John O’Quigley (from 
the University of Leeds) were hired at the Can¬ 
cer Center to replace me when I left for the NSF. 
Lloyd and I laughed that it is not often one’s dis¬ 
sertation advisor replaces his student! John was ini¬ 
tially responsible for implementing a random walk 
rule that I had designed in a pilot study for a bone 
marrow clinical trial. He let them get away with a 
simple dose escalation procedure, but he and Lloyd 
got introduced to the subject at that time. They 


immediately thought of doing a Bayesian alterna¬ 
tive, and it was published in 1990 in Biometrics 
(O’Quigley, Pepe and Fisher, 1990); the major ran¬ 
dom walk paper appeared in Biometrics in 1997 
(Durham, Flournoy and Rosenberger, 1997). Most 
of the Bayesian literature was, by necessity, simula¬ 
tion based, whereas Steve and I were busy obtaining 
a complete workable probabilistic theory of the ran¬ 
dom walk procedures. 

There are a number of philosophical differences 
among the approaches. Fedorov would call the CRM 
a “best intention” approach, because it involves pre¬ 
dicting a target dose and treating the next patient at 
that dose, sequentially. Our approach is estimation- 
motivated. The idea is to get allocations into a re¬ 
gion of interest that allows us to efficiently estimate 
the dose-response curve in that region. 

There is also a short-memory and long-memory 
distinction: allocation probabilities for the ran¬ 
dom walk rule converge exponentially fast to their 
asymptotic limits. Alternatively with best intention 
designs (which to date are long-memory designs), 
nonrepresentative early allocations can cause the de¬ 
sign to converge to the wrong dose (see, e.g., Azriel, 
Mandel and Rinott, 2011; Oron, Azriel and Hoff, 
2011; Azriel, 2012). Such phenomena were observed 
early on in the context of stochastic approxima¬ 
tion designs (e.g., Lai and Robbins, 1982; Bozin and 
Zarrop, 1991). 

Adaptive optimal designs are promising long 
memory designs, but they depend on parameter esti¬ 
mates to get started. Random walk procedures that 
target optimal design points provide good start¬ 
up information with small sample sizes. Alterna¬ 
tively, one can regularize the information matrix, a 
“fix” that is often called “Bayesian designs” even 
though no posterior distribution is obtained. True 
Bayesian estimator updates coupled with dose al¬ 
locations made in some stable optimal way, rather 
than in a “best intention” way are also promising. 

Rosenberger: What is the future of adaptive de¬ 
signs? Do you think all clinical trials will eventually 
be adaptive? 

Flournoy: I think there is a great future for adap¬ 
tive designs. I think experimentation will always in¬ 
volve a series of trials; the question is how well one 
utilizes information from one to the next. There is 
a lot of value in relatively small but sequential tri¬ 
als (see Flournoy, 2014), because these trials involve 
many design features, including the grid size and 
range on which you are operating. The best use of 
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one experiment may be to tell you how you could 
have better selected design characteristics; then you 
can refine the estimate of the target of interest. 

Some of my work has been on inference and es¬ 
timation following adaptive designs (e.g., Rosen- 
berger, Flournoy and Durham, 1997; Ivanova and 
Flournoy, 2001; May and Flournoy, 2009; Lane, Yao 
and Flournoy, 2014). One has to be careful doing 
everything sequentially because some of the interim 
changes may cause final estimates to lack normal¬ 
ity. For example, in best-intention designs, the esti¬ 
mate of a slope parameter can march off to infinity 
for some common models. Also, even if an adaptive 
dose-finding procedure has a fixed total sample size, 
the sample sizes at each dose are random variables. 
In up-and-down procedures, the proportion of sub¬ 
jects allocated to each dose tends to a constant and 
standard asymptotic normality results. But in many 
other adaptive designs, proportions of subjects allo¬ 
cated to each dose tend to a random variable. This 
causes the conditional information matrix to be ran¬ 
dom, even in the limit, in which case standard condi¬ 
tions for asymptotic normality fail. These are many 
interesting questions to be explored about adaptive 
designs. 

5. WOMEN IN STATISTICS 

Rosenberger: Talk about the creation of Pathways 
to the Future, its successes, and its legacy. 

Flournoy: I went to the 1984 Annual IMS Meet¬ 
ing in Lake Tahoe. At that meeting, there were five 


women out of about 200 attendees. It became quite 
clear to me that this was an important place for aca¬ 
demic statisticians to meet and focus on academic 
interests. In anticipation of the 1988 Fort Collins 
IMS meeting, which was separate from the Joint 
Statistical Meetings, I decided it would be great to 
see more women there. So I bounced ideas off Mary 
Ellen Bock (Purdue University) and Lynne Billard 
(University of Georgia). Lynne agreed to take the 
lead in organizing a workshop for women at the up¬ 
coming IMS meeting. Lynne had the brilliant idea 
of having Elizabeth Scott (University of California, 
Berkeley) give the keynote lecture. At that time, 
we were debating whether there was any gender in¬ 
equity in academia, and we weren’t sure. I had never 
experienced problems at UCLA or UW. However, 
when I went to the NSE, Yash Mittal (the first fe¬ 
male director of the probability program) and I saw 
that there were almost no female grantees, and very 
few were even applying for grants. 

The evening presentation by Scott really hit us 
very hard: she had tons of data and randomized 
studies on gender inequity. Any questions about in¬ 
equities in how women were recruited, judged and 
valued were thrown out the window. Scott’s way 
of handling this lecture was wonderful because she 
went through all this horribly depressing data, but 
she then turned around and finished the lecture by 
telling us what we could do to protect ourselves. She 
ended with two positive notes: that outcomes are 
not predetermined, and one can take one’s career in 



Fig. 6. Nancy, husband Leonard, and Lynne Billard at their home in Chapel Hill, NC, 1994- 
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Fig. 7. Nancy, Ingram Olkin and Elizabeth Margosches (formerly with the Environmental Protection Agency) at the Cam¬ 
panile at University of California, Berkeley, 2003. 


one’s own hands. Lynne ran the workshop for the 
next two decades, and she presented Scott’s lecture 
with updated data every year. That lecture was the 
last lecture Scott gave before she passed away. I re¬ 
member well that there was a palpable sigh of relief 
from Scott—that she could turn over her cause to 
the next generation. 

Rosenberger: How did you become NSF program 
director? What was your experience with gender is¬ 
sues there? 

Flournoy: Ingram Olkin has long been a great 
friend and mentor. He is the one who recommended 
me to the NSF for the program director position. I 
was the first female director in the statistics program 
the same year that Yash Mittal was the first female 
probability director. Some people had indicated to 
the division director their fear I was going to give 
all the grant money to biostatistics. I convinced him 
that I could represent the entire statistics field. 

One day I remember answering the phone and a 
professor on the line yelled “I said I wanted to speak 
to the director,” thinking a woman on the phone 
must be a secretary. 

We had a good travel budget and I went to as 
many young women’s lectures as I could. I would go 
up at the end of their talk and ask if they would be 


interested in applying for a grant. By the time I left 
NSF, the proportion of grant proposals from women 
was proportional to their presence in the field. A 
suggestion is such a small thing, and yet clearly im¬ 
portant messages weren’t being transmitted to fe¬ 
male faculty. 

Rosenberger: Was discrimination subtle or not so 
subtle when your career was developing? 

Flournoy: Well, there was always sexist behavior 
and many things that were said and done are consid¬ 
ered inappropriate or even sexual misconduct today. 
When I went on the job market for a fully academic 
position I found that many men were incredulous. 
Some would make outrageous comments directly to 
me as if I were invisible (or a man). Men in my own 
age category were often dismissive or oblivious to 
my presence. Some of the older generation was very 
helpful and supportive (I think of Shanti Gupta, 
Purdue University; Norman Johnson, University of 
North Carolina at Chapel Hill; Lucien LeCam, Uni¬ 
versity of California, Berkeley; Ingram Olkin, Stan¬ 
ford University; and Manny Parzen, Texas A&M 
University). The younger generation just thought of 
me as another senior person, so they were fine. 

Rosenberger: What is your feeling about the role 
of women in statistics today? I can say, from my 
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perspective on 20 years of search committees, that 
from a hiring perspective, we are thrilled to have 
qualified women candidates and compete hard to 
get them. And certainly policies on tenure to allow 
maternity leave have vastly improved over the years, 
as have the composition of committees and senior 
administrators. Is there any work left to be done? 

Flournoy: You can see improvement, but there are 
still troubling facts: just try to find a woman in the 
2013 JSM awards brochure, for instance. Women are 
getting hired at proportional rates now, but awards, 
tenure and advancement are areas where there much 
is left to be done. See Lynne Dillard’s new update of 
Scott’s old data on the subject (Dillard and Kafadar, 
2015). That will depress you. 

6. CONCLUSION 

Rosenberger: You talked a little about your tran¬ 
sition into a fully academic position. The latter part 
of your career was spent at AU and University of 
Missouri (MU), and considerable time as depart¬ 
ment chair, and a mentor to many diverse students. 
Talk about this. 

Flournoy: AU was a great place for me when I 
went there in 1988. I had left the Cancer Center 
with a staff of 23, a budget of $700,000 and re¬ 
sponsibilities that had become a burden when I be¬ 
came convinced of the need for more nimble learning 
strategies in dose-finding clinical trials. I had eight 
doctoral students at AU, and all but two of them 
developed mechanisms to control random walks and 
urn models, and to provide mathematical descrip¬ 
tions of their controlled behavior. One worked on 
issues of inference following an adaptive design and 
one worked on a problem in economics. I am proud 
that four of these students are black and two are 
women. 

Unfortunately, a very destructive president came 
to AU, and by 2000 it was clear that STEM gradu¬ 
ate programs were going to be dismantled. AU had 
one of the oldest statistics doctoral programs in the 
country and it was sad to see it threatened by ig¬ 
norance and arrogance. To remain in a department 
with a doctoral program, I needed to move and this 
led me to accept the chair at Missouri in 2002. When 
I stepped down as chair in 2011, I had doubled the 
number of tenure-track faculty and added five teach¬ 
ing faculty positions. I increased the presence of the 
department across campus through joint appoint¬ 
ments and a targeted increase in service courses, and 



Fig. 8. Nancy near Aasgard Pass in the Enehantment Lakes 
Wilderness Area, Washington, where she was hiking with her 
husband Leonard and her colleague Lori L. Thombs (Univer¬ 
sity of Missouri) following the 2006 Joint Statistical Meetings 
in Seattle. 

I increased the prestige of the department nation¬ 
ally, personally promoting our faculty and enabling 
their participation in national and international ac¬ 
tivities. More details can be found in a Chapter I 
recently wrote on the history of statistics at MU 
(Flournoy and Galen, 2012). 

I have graduated seven doctoral students from 
MU. We worked on adaptive and optimal designs; 
we developed new models for specific, challenging 
dose-response problems and we have illuminated the 
effect of having dose allocations depend on the his¬ 
tory of prior allocations and responses. My students 
continue to bring me a great deal of pleasure. 

Rosenberger: What are your hobbies and inter¬ 
ests? 

Flournoy: I love hiking. I am not happy with a 
trip that takes less than four days. A four-day trip 
has two days out and two days back—so one is never 
very far from a road. After hiking for more than two 
days, one must rely on one’s self much more com¬ 
pletely. It is so peaceful. I gave up trying to hike 
in the East and the Midwest United States. One 
just can’t get far enough away from roads; and the 
mountains aren’t high enough. I like trekking around 
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timberline for a week or more where the views are 
spectacular. I keep going back to Yosemite, Kings 
Canyon and Sequoia National Forests. Nepal was 
great, too. I try to get in one long hike each year. 
In the meantime, I dance. I resumed ballet classes 
while at AU; it is great mind-to-body exercise and 
wonderful for strength and balance. Leonard and I 
enjoy English country dance together. Throw in Pi- 
lates and yoga and I am happy. 

To survive a severe health challenge that had the 
doctors stumped, I gained considerable knowledge 
of alternative methods and became accomplished in 
some. But that is another story. 

Rosenberger: What’s next for Nancy Flournoy? 
Flournoy: Well I have a lot of ideas. I’m really in¬ 
terested in questions of inference following adaptive 
designs. We have some examples in two stage designs 
that maximum likelihood estimators are mixtures of 
normals; some designs lead to estimators that are 
normal with random variances. I think our prelim¬ 
inary results are generalizable, but this remains to 
be shown. I’m optimistic that tractable solutions to 
seemingly intractable problems are at hand. 
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